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METHOD AND APPARATUS FOR PHOTOGRAPHING /RECOGNIZING 

A FACE 

BACKGROTTNn oftntt t nwntton 
1. Technical field 

The present invention relates to method and apparatus for 
photographing/recognizing a face which can inteUigently photograph an 
object captured by a camera. Particularly, the present invention relates 
to method and apparatus for photographing/ recognizing a face which may 
distinguish a specific person by photographing and recognizing a face 
portion of the person accurately through adjusting the camera according 
to motion of the object, and generate alann to a dangerous character so to 
be appUed to a security system, an image recognizing system, a picture 
telephone, an internet on-line game and a virtual reality system, etc. 

2. Description of the Prior Art 

In order to photograph a face, a conventional manless monitoring 
or face recognizing system employs at least one monitoring camera 
mstalled in a desired position such as ceiling or wall. With use of image 
signals from the camera, the conventional system enables to remotely 
monitor a specific field or manlessly record whole scene of the field. Such 
systems are usually used forjudging condition of a specific field. 

However, such manless monitoring system or face recognizing 
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system should capture image signals of the field from a designated 
position (such as height or distance). Therefore, the image signals are 
irregular due to variation of height of an object or distance between the 
object and the camera. Furthermore, an object or face which is hid or 
5 moves fast to get away the camera can be hardly recognized. 
Recognizable image signals can be just obtained by photographing the 
object in front or artificially adjusdng the camera with use of a remote 
controller or a direction adjusting key. 

Therefore, the conventional system has trouble to recognize a face 

10 in moving pictures with use of such unrecognizable image data. Because 
it is very difficult to find an intact face image through retrieving a stored 
data, the image data obtained as above cannot provide sufficient evidence 
when security problem happens. In addition, because the stored data is 
in type of moving pictures, it needs a high-capacity storing unit for storing 

15 such moving pictures, unnecessarily. 

In order to overcome such problems, there is needed to develop • 
technique for photographing a face image in front more accurately by 
moving the camera according to motion of the object. 

20 SUMMARY O P THR TNVKNTTON 

The present invention is designed to solve such problems. An 
object of the present invention is to provide method and apparatus for 
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photographing a face which can photograph a face image in a still picture 
by detecting skin color and motion of an object captured by a camera, 
automatically selecting most appropriate object, and making the camera 
track the object automadcally without any artificial manipulation, and 
5 method and apparatus for recognizing a face which can accurately 
recognize a specific person by analyzing the face image of the captured 
object. 

Another object of the present invention is to provide method and 
apparatus for recognizing a face which can cope with any possible 

10 emergency in advance by identifying a face having risk factors (such as 
strange mask, dark sunglasses, pressed-down cap, etc.) through analyzing 
the face image signals of the selected object, and generating alarm to a 
dangerous person. 

In order to obtain above object, one embodiment of the present 

15 invention provides a method of photographing a face comprising: initial 
analyzing step for idendfying position of the face by sampling face data 
from image signals inputted from a camera installed in a predetermined 
position, and then selecting check area in order to track the identified face; 
pan/tilt or zoom control step for moving the camera to right/left, up/down 

20 or forward/ backward direction in order to track the identified face in the 
selected check area; determining step for determining whether the face is 
existing or abnormal by sampling and analyzing contour of the face 
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identified in the check area; and storing and transmitting step for storing 
image of the determined face or transmitting the image to another 
recognizing system. 

In the embodiment, the method of photographing a face may further 

5 include the step of generating alarm about abnormal face after analyzing 
and determining the contour of the face sampled in the check area. 

In order to accomplish the above objects, another embodiment of the 
present invendon provides a method for recognizing a face comprising: 
initial analjrzing step for identifying position of the face by sampling face 

10 data from image signals inputted from a camera installed in a 
predetermined position, and then selecting check area in order to track the 
identified face; pan/ tilt or zoom control step for moving the camera to 
right/left, up/down or forward /backward direction in order to track the 
idendfied face in the selected check area; determining step for determining 

15 whether the face is existing or abnormal by sampling and analyzing 
contour of the face identified in the check area; and composing a personal 
mask by generating 3-dimensional informadon for a face image in the 
check sirea in case that the face is existing, and then recognizing a specific 
person by comparing the personal mask with a previously stored personal 

20 mask. 

For obtain the above objects, still another embodiment of the 
present invendon provides a method for recognizing a face comprising the 
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steps of: selecting a plurality of check areas about image signals captured 
by a camera, selecting a specific check area by determining whether a face 
exists in the corresponding check area, and adjusting direction of the 
camera according to motion of an object in the selected check area; 
5 rearranging images by detecting characteristics about image signal where 
a face is existing in the check area after determining whether a face exists, 
generating 3-dimensionaJ information according to displacement between 
the rearranged images, and composing a face mask of the object with use 
of the 3-dimensional information; and determining whether image for a 

10 specific person is registered or not by searching the composed face mask 
from personal registration masks in a database. 

For accomplish the above objects, another embodiment of the 
present invention provides a method for recognizing a face comprising the 
steps of: selecting a plurality of check areas by respectively sampling skin 

15 color data and motion data from image signals inputted through a camera, 
selecting a specific check area by retrieving a specific check area by 
inspecting face components about image signals in the corresponding 
check area, rearranging image by retrieving characteristics of the selected 
check area, and adjusting direction of the camera with determining 

20 whether a face exists or not with use of the rearranged image; rearranging 
images about image signals where a face is existing in the check area, 
generating 3-dimensionaI infonnation according to displacement between 
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the rearranged images, and composing a personal face mask by comparing 
the 3-dimensionaI information with a reference mask; and determining 
whether image for a specific person is registered or not by searching the 
composed face mask from personal registration masks in a database. 
5 For accomplish the above objects, the present invention also 

provides an apparatus for photographing a face comprising: image input 
means for capturing image of an object photographed by a camera 
installed in a predetermined position, and receiving serial image signaJs; 
pan/tilt or zoom driving means for moving the camera of the image input 

10 means to right/left, up/down or forward/backward direction; main control 
means for controlling the pan/tilt or zoom driving means such that the 
camera tracks motion of the object, selecting a check area by detecting 
skin color and motion of the object from the image signals inputted from 
the image input means, retrieving face portions from the image signals in 

15 the selected check area, and storing and displaying the face portions; 
storing or retrieving means for storing the image signals from the main * 
control means or retrieving face portion from the image signals; and 
display means for outputting the image signal corresponding to the face 
portion outputted from the main control means. 
20 In the embodiment, the apparatus for photographing a face may 

further include an alarm generating unit for generating alarm about 
abnormal face after analyzing and determining the contour of the face 



6 



sampled in the check area. 



BRIEF nFRCPTPTTON OF THF nRAWTNrr<^ 

These and other features, aspects, and advantages of the present 
5 invention will become better understood with regard to the following 
description, appended claims, and accompanying drawings, in which like 
components are referred to by like reference numerals. In the drawings: 

FIG.- 1 is a schematic block diagram for showing configuration of 
method for photographing/ recognizing a face according to one embodiment 
10 of the present invention; 

FIG. 2 is a functional block diagram for showing a main controller 
shown in FIG. 1; 

FIG. 3 shows signal flow of an image input process; 

FIG. 4 shows signal flow of an initial analyzing process; 
15 FIG. 5 shows signal flow of a determining process; 

FIG, 6 is a reference showing a screen and a screen movable position 
for illustrating a pan /tilt or zoom control process; 

FIG. 7 is a reference for illustrating a process of photographing an 
object in center of the screen at an appropriate distance when an object is 
20 captured in a photographing region; 

FIG. 8 is a functional block diagram for illustrating an image storing 
and transmitting process; 
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FIG. 9 shows a face template (a) and an input image (b) used for 
retrieving a face portion; 

FIG. 10 is a flow chart for illustrating outline of the step of 
processing image signals in order to explain the method for 
5 photographing/ recognizing a face according to one embodiment of the 
present invention; 

FIG. 11 is a flow chart for illustrating a face data normalizing 
process; 

FIG. 12 shows contour sampling arrays of the face data; 
10 FIG. 13 is a flow chart for illustrating the step of adjusting strength 

of the contour of the face data according to difference in brightness; 
FIG. 14 shows a template around eyes; 

FIG. 15 shows a template (a) and contour data (b) around mouth; 
FIG. 16 is a block diagram for illustrating method for 
15 photographing/ recognizing a face according to another embodiment of the 
present invention; 

FIG. 17 is a flow chart for illustrating the method for 
photographing/ recognizing a face implemented in FIG. 16; and 

FIG. 18 is a flow chart for illustrating another method for 
20 photographing/ recognizing a face implemented in FIG. 16. 

DRTAn.Fn nR.qrRiP TTON of th e preferrkd emrodtmrnt 
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Hereinafter, preferred embodiments of the present invention will be 
described in detail with reference to the accompanying drawings. 

FIG. 1 to FIG. 14 are block diagrams of circiiit configuration and 
flow charts of signal process for illustrating face 
photographing/ recognizing method and apparatus according to one 
embodiment of the present invention. FIG. 16 to FIG. 18 are a block 
diagram and flow charts for illustrating face photographing/ recognizing 
method and apparatus according to another embodiment of the present 
invention. 

At first, one embodiment of the present invention is described in 
detail referring to FIG. 1 to FIG. 15. 

FIG. 1 is a schematic block diagram for showing circuit 
configuration for implementing the apparatus for 
photographing/ recognizing a face. 

As shown in the figure, the apparatus of the present invention 
includes an image input unit 10 for receiving serial image signals by 
capturing image of an object photographed by a camera installed in a 
predetermined position. The apparatus also includes a pan/ tilt or zoom 
driving unit 20 for moving the camera of the image input unit 10 to 
right/left, up/ down or forward /backward direction. The apparatus also 
includes a main controller 30 for controlling the pan/ tilt or zoom driving 
unit 20 such that the camera tracks motion of the object. The main 
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controller 30 also selects a check area by detecting skin color and motion 
of the object from the image signals inputted from the image input unit 10. 
The main controUer 30 also retrieves face portions from the image signals 
in the selected check area, and then stores and transmits the face portion 
5 of the image signals to a display discussed below. The apparatus is also 
provided with a storing or retrieving unit 50 for storing the image signals 
from the main controUer 30 or retrieving face portion from the image 
signals. The display 40 is of course included in the apparatus for 
outputting the image signal corresponding to the face portion outputted 
10 from the main controller 30. 

In addition, the apparatus of the present invention may include an 
alarm generating unit 60 for generating alarm by using a visual display, 
an audio device according to results of face retrieval from the main 
controller 30. 

15 The image input unit 10 is installed to be movable to a up/down, 

right/left or fopvard/backward direction with use of the pan/tilt or zoom 
driving unit 20 mounted outside. In addition the image input unit 10 
includes a camera (e.g. a digital camera in the present invendon) for 
converting the image captured in lens into an image signal which is 
20. defined in a format for digital process (e.g. RGB or YUV). The image input 
unit 10 also may include an image processor for processing the image 
signal in a regular frame intenral and then providing serial digital image 
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data (e.g. 15-30 frames per second). 

FIG. 2 is a block diagram for showing functional inner configuration 
of the main controller 30 of FIG. 1. As shown in the figure, the main 
controller 30 includes an initial analyzing unit 31 for selecting a check 
area by sampling portions which show skin color or motion from the input 
images from the image input unit 10. The main controller 30 also 
includes a determining unit 32 for sampling contour of the face from the 
check area, comparing the contour with a predetermined face template, 
determining whether the face is existing or abnormal according the 
comparison, and then selecting an object for tracking. The main 
controller 30 includes an object managing unit 33 for determining whether 
the object for tracking selected by the determining unit 32 is positioned in 
center of photographing region, and then calculating and outputting 
adjustment value of the camera for tracking motion of the corresponding 
object. The main controller 30 also includes a pan/tilt or zoom control 
unit 34 for generating a pan/ tilt or zoom control signal in order to move 
the image input unit 10 to a right/left, up/down or forward /backward 
direction according to the adjustment value of the object managing unit 
33. In addition, the main controller 30 includes an image storing and 
transmitting unit 35 for storing face image captured by the image input 
unit 10 and transmitting the image to the display 40 and the storing or 
retrieving unit 50, respectively. The main controller 30 also includes a 
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face recognizing and retrieving unit 36 for recognizing and retrieving the 
stored face image, and then transmitting the results to the alarm 
generating unit 60. 

The initial analyzing unit 31, as shown in FIG. 3, detects a moving 
portion S316 by comparing a current image data S314 sampled from an 
input image signal S3 13 with image data S3 15 previously stored. The 
initial analyzing unit 31 then detects a portion, of which proportions of 
color elements (e.g. red R, green G, blue B or YUV) are in range of skin 
color, in the image signal from the image input unit 10, 8317. The initial 
analyzing unit 3 1 then selects a portion where both skin color and motion 
are detected as a check area, S3 18. - • 

The determining unit 32, as shown in FIG. 4, samples a contour of 
face in the check area D selected from a camera photographing region S 
by the initial analyzing unit 31, S41 1. After that, the determining unit 32 
converts size of a previously stored face template into that of the check 
area D, and then compares the template with the contour. Then, if a ' 
contour approximate to the face template is detected in the check area D, 
the determining unit 32 recognizes that the face exists in the check area 
D, S412. 

In addition, as shown in FIG. 5, after determining that a face exists 
by comparing the template with the contour S51 1, the determining unit 
32 determines whether there is a previous object for tracking, S512. After 
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the detennination, the detennining unit 32 selects an object for tracking 
with results of the comparison between the currently.selected. image data 
and the previously stored image data, S513. S514. At this time, in case 
that two or more objects for tracking are detected, the determining unit 32 
selects an image most approximate to the previously stored object for 
tracking as a new object for tracking, S514. If there is no previously 
object for tracking, the determining unit 32 selects an image nearest to 
center of the photographing region as a new object for tracking, S513. 

The object managing unit 33. as shown in FIG. 6. reads location (SX, 
SY) of a screen in whole region which can be photographed with the 
camera and then detennines whether the object of tracking selected by the 
detennining unit 32 is positioned in center of the camera photographing 
region (x, y). At the same time, the object managing unit 33 calculates 
location (MX, MY) to which the screen will move, about the whole area 
(Pmax, Tmax) which can be ranged by the camera through pan/tilt control, 
and then outputs camera adjustment value to the paji/tilt or zoom control ' 
unit 34. 

As shown in FIG. 7, the pan/tilt or zoom control unit 34 generates 
a pan/ tilt or zoom signal in order to move the camera of the image input 
unit 10 to a right/left, up/down or forward/backward (toward 
paper/opposite paper in the figure) direction according to the camera 
adjustment value from the object managing unit 33. If an object (d) is 
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detected by the camera as shown in FIG. 7 (B), the pan/ tilt or zoom 
control unit 34 positions the object (d) to center of the screen as shown in 
FIG. 7 (C). 

As shown in the signal flow of FIG. 8, the image storing and 
5 transmitting unit 35 converts face image S611 captured by the image 
input unit 10 into a regular size S612, and then stores the face image in 
shape of bit map picture BMP S613. The image storing and transmitting 
unit 35 then stores the face image in a small format type (e.g. JPEG) for a 
predetermined long time. In addition, the image storing and transmitting 
10 unit 35 transmits the image of the object for tracking to the display 40 for 
displaying the image on the screen S615, or transmits to the face 
recognizing and retrieving imit 36 for recognizing the face S616. 

The face recognizing and retrieving xmit 36 retrieves characteristics 
of a person in order to find a face of the peculiar person from the input 
15 face data. The face recognizing and retrieving unit 36 then outputs a face 
image data corresponding to the characterisdcs (same characteristics of • 
the face template of FIG. 9 (a) and the face image of FIG. 9 (b)). 

FIG. 10 is a flow chart for illustrating outline of an image signal 
process for explaining one embodiment of the face 
20 photographing/ recognizing method according to the present invention. 

As shown in the figure, the face photographing/ recognizing method 
according to one embodiment of the present invention includes initial 
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analyzing steps S101-S109, determining steps S111-S119, and an image 
storing and transmitting step S121. The initial , analyzing steps S 101- 
S109 selects a check area by sampling portions having skin color and 
motion from the input image photographed through the image input unit 
5 10 installed to a predetermined position. The determining steps SI 11- 
S 1 19 selects an object for tracking by sampling a face contour from the 
selected check area, comparing the contour with a predetermined face 
template, and then determining whether the face is existing or abnormal 
according to the comparison. The image storing and transmitting step 
10 S121 recognizes and retrieves the face image captured by the image input 
unit 10, and then stores the inputted face image and the contour of the 
object for backing sampled from the face image. 

Though not shown in the figure, another embodiment of the present 
invention may include an object managing step and a pan/tilt or zoom 
15 control step. The object managing step determines whetiier the object for 
tracking selected in the determining step is positioned in center of the 
camera photographing region, and tiien calculates a camera adjustinent 
value for backing motion of the corresponding object for ti-acking. The 
pan/tilt or zoom conti-ol step' controls pan/tilt or zoom function in order 
20 to move the image input unit 10 to a right/left, up/down or 
forward/dou-nward direction according to the camera adjustinent value 
calculated in the object managing step. 
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Still another embodiment of the present invention may further 
include an alarm generating step S120 for generating alarm when the face 
is abnormal after analyzing and retrieving the face contour in the sampled 
check area. 

In addition, further another embodiment of the present invention 
may include the steps of generating 3-dimensional information for a face 
image in the check image in case that the face is existing in the check 
area, - composing a . personal mask with use of the 3-dimensional 
information, and then recognizing a specific person by comparing the 
composed personal mask with person masks previously stored in, such as, 
database, such that it can be applied to face image recognition. 

F\irthermore, the present invention may store an instantaneous face 
image (face image in a still picture) which is accurately photographed in 
the check area through the initial analyzing step and the pan /tilt or zoom 
control step in a separate storing unit such that the instantaneous face 
image can be applied to an image recognizing device for a investigation ' 
purpose, a security device designed for regulating machines by recognizing 
a face, and so on. 

FIG. 11 is a flow chart for illustrating a face data normalizing 
process with arrays of eyes, FIG. 12 shows contour sampling arrays of 
the face data. FIG. 13 is a flow chart for illustrating the step of adjusting 
strength of the contour of the face data according to difference in 
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brightness. FIG. 14 shows a template around eyes and FIG. 15 shows a 

template (a) and contour data (b) around mouth. 

Operation and effect of the present invention will be explained 
referring to the figures. 
5 Referring to FIG. 11, an inclination is examined by using 

arrangement of the eyes retrieved from the face data. S711 . The face data 
is then normalized S712, so that a face data having a regular size and 
shape can be sampled S713, 

Referring to FIG. 13, at first, characteristics of the contour in eyes, 
10 nose, mouth, eyebrows of an input face data are sampled S811. 
Arrangements of the sampled contour are substituted with each pixel 
S812. In addition, in order to determine strength of brightness, an 
average brightness is calculated from the input face data S813. After 
that, the average brightness is compared with, brightness of the pbcels 
15 S8 14 so to generate a horizontal contour sampling data S815, 

- At first, a digital image inputted from the camera of the image input 
unit 10 is defined in a format for digital process (e.g. RGB or YUV) and 
provides resolution (e.g. having 640x480 pixels) required to the image 
processor. At this time, the current input image is compared with a 
20 previous input image, and then updated as a reference image (another 
previous input image for next input image). In addition, a check area is 
selected by detecting skin color and motion of an object for tracking 
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through comparison between the current input data and the previous 
input data. 

After that, the initial analyzing unit 31 of the main controller 30 
selects a range for retrieving by detecting a portion having skin color 

5 consisted of RGB proportions and motion through image sampling of the 
image data from the image input unit 10. The initial analyzing unit 31 
stores the data after detecting the portion and motion, and then uses the 
data for detecting motion of next image. In addition, the initial analyzing 
unit 3 1 selects check area by using motion and transmits the check area 

10 to the determining unit 32. 

The determining unit 32 then determines whether the face is 
existing by retrieving the check area analyzed in the initial analyzing unit 
31. On the purpose of that, the determining \init 32, at first, detects a 
contour where a center portion is dark and upper and lower portions are 

15 bright from the image signal of the check area as shown in FIG. 12. As a 
result of the contour detection, a contour S815 in which eyebrows, eyes, 
lower part of nose and mouth are outstanding can be obtained. The 
determining unit 32 then determines the face existing in the check area 
in case that the contour in the check area indicates higher value than a 

20 reference value through comparison with a previously stored face template. 
Because the check area does not have a regular size, size of the 
previously stored template should be changed before the comparison. 
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After that, the object managing unit 33 selects an object for tracking 
by comparing a current sampled face data with a previous-data. If one 
face is detected in a new screen; the face becomes an object for tracking, 
while two or more faces are detected, one most approximate to the ' 
5 previous object is selected as an object for tracking. If there is no 
previous object for tracking, a face nearest to center of the screen becomes- 
selected as an object for tracking. 

Next, the pan/tilt or zoom control unit 34 is for operating the 
camera to track the object in case that .the object for tracking is not 
10 positioned in center of the camera. The pan/tilt or zoom control unit 34 
includes the pan/tilt or zoom driving unit 20 attached to the camera of the 
image input unit 10 such that the camera can be operated by commands 
from the pan/tilt or zoom control unit 34 of the main controller. The 
pan/ tilt or zoom driving unit 20 is connected to the pan/ tilt or zoom 
15 control unit 34 through serial, parallel, universal serial bus USB, or 
wireless communication. 

After retrieving the input face images, the image storing and 
transmitting unit 35 stores an input face image in a separate storing unit 
in a picture type, and transmits the image together with the characteristics 
20 sampled for a recognizing system. 

The face image retrieving unit 36 detects characteristics of a specific 
person in order to find a face in the input face data. The face image 
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retrieving unit 36 then outputs a face image data corresponding to the 
characteristics. 

At this time, the processed data (input data) is a rectangular image 
data in which only face portion is sampled. Size of the data may be 
5 dilTerent each other according to size of a face, and inclination of the data 
is not aligned. 

Therefore, the process for checking a face, at first, retrieves eyes with 
characteristics of face components with use of the face data, as shown in 
FIG. 10, S 101, SI 03. Then, the process normalizes the face data with use 
10 of arrays of the eyes in order to make size and shape of the face uniformed 
as shown in FIG. 1 1. S105. 

After that, the process samples a contour of each component of the 
face having \iniform size and shape S107, S109, and then retrieves eye 
portions with use of an eye template Sill so to determine error by 
15 examining the retrieved eyes in detail 8113. 

Sequentially, the process retrieves a mouth portion with use of 
mouth and mouth surrounding templates SI 15, and therefore determines 
error through detail examination for the retrieved mouth SI 17. 

In addition, with use of nose and nose surrounding templates and 
20 eyebrows and eyebrow surrounding templates (not shown in the figures), 
the process may retrieve a nose portion and eyebrow portions in order to 
determine error through detail examination for the retrieved nose and 
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eyebrows. 

Then, the process synthetically determines error ^ath use of results 
from the examination for the eyes and the mouth (possibly, nose and 
eyebrows), S119. 

5 If there is no error as a result of the eye, nose, mouth, eyebrow or 

synthetic determination, the process writes and stores a face picture of the 
person and trajismits the retrieved data to a door managing system or a 
passenger identifying system with use of image recognition, S121. 
Therefore, if any problem is occurred, a problematic person can be easily 
10 retrieved with use of any retrieving program. 

On the other hand, if there is error as a result of the eye, nose, 
mouth, eyebrow or synthetic determination, the process determines the 
person having a risk factor, for example with a mask, dark sunglasses, a 
pressed-down cap. In this case, the process generates alarm such that a 
15 supervisor or a serviceman may cope ^.-ith the emergency situation, S120. 

The present invention as described above can accurately photograph ' 
a face in a rapid speed with use of a camera installed in an entrance door, 
which enables to write a face of a passenger. In addition, capacity of the 
storing unit can be remarkably reduced in fact that the present invention 
20 stores only a still picture instead of moving pictures which are employed 
in the prior art. 

Moreover, in case that an abnormal person who enters with a mask. 
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dark sunglasses or a cap pressed down, that is, a person having a risk 
factor appears, the present invention generates alarm to a supervisor or a 
serviceman in advance in order to cope with an emergency situation. 

In addition, because a camera which tracks motion of a face is 
5 installed in a door, the present invention can make criminals cowered, 
which can prevent crimes more effectively than any other supenasing 
systems. 

Furthermore, the present invention may overcome inconvenience of 

the prioi: art which needs to show a face to a fixed camera because the 
10 camera intelligently moves to an up/down or right/left direction in order 

to show a face in front accurately in case of using such as picture phone. 
In addition, the present invention provides a more convenient 

function because the camera can correspond to a tall or little person, 

which is impossible for the prior art. 
1 5 The present invention also has much more advantages in that the 

present invention may identify a passenger in connection with a door 

system, or can be used to an animation manufacturing or an on-line real 

time simulation game using automatic image tracking. 

20 Hereinafter, another embodiment of the face recognizing method 

according to the present invention will be described with reference to FIG. 
16 to FIG. 18 in detail. 
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FIG. 16 is a block diagran: for illustrating configuration of a face 
photographing/ recognizing system for implementing the face recognizing 
method according to another embodiment of the present invention. 

As shown in the figvire, an image input unit 100 photographs an 
object through a plurality of cameras toward predetermined directions. 
In general, the image input unit 100 employs at least two CCD cameras 
110. 

A preprocessing unit 200 receives an image signal from the image 
input unit 100. The preprocessing unit 200 eliminates noise in the image 
signal and then samples a contour about the image signal. 

A characteristic sampling unit 300 samples characteristics of the. 
image signal on the basis of the conto\ir of the image signal inputted from 
the preprocessing lanit 200. 

A 3-dimensional information sampling unit 400 rearranges image 
by rotating, zooming or moving the image signal from the preprocessing 
unit 200 with use of the characteristics sampled by the characteristic 
sampling unit 300, and then samples 3-dimensional information with use 
of displacement information between the rearranged images. 

A mask generating unit 500 compares the 3-dimensional 
information sampled from the 3-dimensional information sampling unit 
400 uith a predetermined reference mask, and then generates a new mask 
peculiar to each person. 
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A database 600 includes a plurality of image data obtained from 
image signals of proper users as a masking data. 

A shape recognizing unit 700 is generally a central processing unit 
CPU. The shape recognizing unit 700 compares the new mask generated 
5 in the mask generating unit 500 with the masking data in the database 
600, and then determines whether the new mask is same as the masking 
data. At this time, if there is a mask same as the new mask in the 
database 600, the current image photographed through the camera is 
recognized as a previously registered mask and the user is classified as a 
10 proper user. Or else, the image is recognized as an iinregistered mask 
and the user is classified as an improper user, 

A storing media 800 includes an operating program for overall 
system and a plurality of data required for recognizing faces. Such 
storing media 800 operates according to a control signal from the shape 
15 recognizing imit 700 so to output the previously stored data to the shape 
recognizing tinit 700. The storing media 800 can be a card having 
semiconductor memory, a hard disk,, an optical disk, etc. 

The image input unit 100 may include the CCD camera 1 10 for 
photographing an object, an image capturing unit 120 for capturing signal 
20 from the camera in order to display the signal in a designated type, a 
camera driving control unit 130 for outputting a camera driving control 
signal by calculating a current rotating range of the CCD camera with use 
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of a check area selected from the characteristic sampling unit 300, and a 
camera driving unit 140 for rotating the camera to. a specific direction 
according to the camera driving control signal of the camera driving 
control unit 130. 

FIG. 17 is a flow chart for illustrating the face 
photographing/ recognizing method according to another embodiment of 
the present invendon. As shown in the figure, the method includes steps 
S200-S250 of selecting a check area for an image signal captured through 
the camera, detecting characteristics from the corresponding check area, 
rearranging the image, detecting a portion where a face is existing through 
retrieval of face components, sampling and examining characteristics of 
the portion having a face, and then adjusting direction of the camera with 
determining whether a face exists. The method further includes a step 
S260 of rearranging again the rearranged image about the image signal 
having a face in the check area by the characteristics, generating 3- 
dimensional information according to displacement information between 
the rearranged images, and the composing a face mask of the object with 
use of the 3-dimensional information. The method also further includes 
a step S270 of determining whether image for a specific person is 
registered or not by searching the composed face mask from personal 
registration masks in the database. 

FIG. 18 is a flow chart for illustrating the face 
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photographing/recognizing method according to another embodiment of 
the present invention. The method includes steps S300-S340 of selecting 
at least one check areas by respectively sampling skin color data and 
motion data from image signals inputted through cameras, matching the 
template with the face components with use of brightness difference in the 
corresponding area, and adjusting direction of the cameras after 
determining whether a face exists. The method aJso includes steps S350- 
S370 of rearranging images about image signals where a face is existing 
in the check area, generating 3-dimensional information according to 
displacement between the rearranged images, and composing a personal 
face mask by comparing the 3-dimensional information with a reference 
mask. The method also further includes steps S380-S400 of determining 
whether image for a specific person is registered or not by searching the 
composed face mask from personal registratipn masks in a database. 

Each embodiment of the present invention as constructed above is 
operated as follows. 

r 

At first, operation of an embodiment in FIG. 17 is explained with 
reference to the block diagram in FIG. 15, 

The image input unit 100 receives image signal from the CCD 
camera 1 10 and the image capturing unit 120. The preprocessing unit 
200 eliminates noise, and then executes contour correction and filtering. 

The image signal processed in such a manner is then inputted to the 
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characteristic detecting unit 300. The characteristic detecting unit 300 
generates motion data by detecting motion . of an object through 
comparison between a previous image and a current image of the image 
signal. The characteristic detecting unit 300 generates a skin color data 
5 by detecting a portion in which proportions of color elements (e.g. R(red), 
G(green), B(bluc) or YUV) are in range of skin color. 

The characteristic detecting unit 300 selects at least one check areas 
from a image signal from a field inputted from the image input unit 100 
and the preprocessing unit 200 with use of the motion data and the color 
10 data generated as above, S210. The characteristic detecting unit 300 
selects a region where a face is existing by identifying the face components 
from the selected check area S210-1, and then rearranges the input image 
from the CCD camera 1 10 and the preprocessing unit 200 with use of the 
characteristics, S220. 
15 The motion and skin color data of the generated check area, the 

detected characteristics, and the rearranged image data of the check area 
are inputted to the 3-dimensional information sampling unit 400 and the 
camera driving control unit 130, respectively. 

The camera driving control unit 130 compares the motion and skin 
20 color data of the generated check area, the detected characteristics, and 
the rearranged image data with previously inputted and processed check 
area data. 
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The camera driving control unit 130 also calculates moving distance 
of the check area by comparing the data of two check area as described 
above, S230. In addition, bjnrotating the CCD camera 1 10 as much as 
the moving distance, rotating distance and angle are determined in order 
5 to photograph the object again. 

The determined rotating distance and angle of the CCD. camera 1 10 
are then inputted as a camera driving control signal to the camera driving 
unit 140. The camera driving unit 140 receiving the camera driving 
control signal then rotates the CCD camera 110 as much as the 
10 predetermined rotating angle to a certain direction, S240. Such camera 
control operation is repeated whenever a new object is detected. 

On the other hand, the 3-dimensional information sampling unit 
400 receiving the motion and skin color data of the check area, the 
detected characteristics, and the rearranged image data from the 
15 characteristic detecting unit 300 then rearranges the image one more time 
with use of the detected characteristics, and then determines whether a' 
face is existing in the check area with use of the rearranged characteristics 
of the image, S250. 

As a result of the determination, 3-dimensional information for an 
20 image signal having a face in the check area is generated witii use of the 
displacement information between a plurality of the rearranged images. 
In addition, by comparing the generated 3-dimensional information with 



28 



a reference mask in the mask generating unit 500, a new personal mask 
is generated, S260. 

Then. The new generated mask is compared with a mask data in the 
database 600 in order to determine whether same mask exists in the 
5 database. If same mask data exists in the database 600, the new mask 
is recognized as a registered mask and then the object currently captured 
through the camera is classified as a proper user. On the other hand, if 
■ there is no same mask in the database 600, the new mask is recognized 
as an unregistered mask and then the object is classified as an improper 
10 user, S270. 

Operation of the embodiment in FIG. 18 is explained in conjunction 
with the block diagram in FIG. 16. 

When an image signal is inputted through the CCD camera 1 10 and 
the image capturing unit 120 of the image input unit 100 and the 
15 preprocessing unit 200, the characteristic detecting unit 300 determines 
the input signal and then compares a current screen with a previous . ' 
screen of the image signal. According to the comparison, motion of an 
object can be detected to generate a motion data, and a portion in which 
proportions of color elements (e.g R. G. B or YUV) are in range of skin 
20 color is detected to generate a color data, S3 1 0. 

In addition, the characteristic detecting unit 300 selects at least one 
check area from the image signal on one field with use of the color data 
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and the motion data generated in the above process, S320. 

When the check areas are selected, the method of this embodiment 
detects a portion where a face is existing through examination of face 
components S320-1, and then picks out the object for tracking in the 

5 corresponding area. The image is rearranged by the characteristics 
detected by the characteristic detecting unit 300 S330, and then the 
present invention identifies presence of the object for tracking in the 
corresponding area from the rearranged image S340. If there is a face of 
the object for tracking in the corresponding area, the image is rearranged 

10 after fmely adjusting the camera such that the image signal having a face 
in the corresponding area is positioned in center of the screen, S350. 
Then, 3-dimensional information is generated with use of displacement 
information among a plurality of the rearranged images S360. The 3- 
dimensional information is compared with a reference mask in the mask 

15 generating unit 500 so to generate new masks each of which is personally 
different, S370. 

If there is no accurate face, the camera is adjusted to return to a 
normal position (ordinary photographing position) by calculating a camera 
adjustment value indicating amount of the camera to move, S34 1 . Then, 
20 the method of this embodiment executes a step S3 10 of generating skin 
data and motion data with receiving the image signal. 

The new mask generated as above is compared with a mask data in 
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the database 600 so to determine whether the new mask exists in the 
database S380. As a result of the determination,, if-same mask exists in 
the database 600, the new mask is recognized as a registered mask S390 
and the object currently captured by the camera is classified as a proper 
user. If there is no same mask in the database 600, the new mask is 
recognized as an unregistered mask, and the object becomes classified as 
an improper user. 

Therefore, the present invention described above identifying a face 
of a person by sampling skin data and motion data from an image signal 
may repeatedly adjust position of the camera until recognizing the face 
fully by rotating the camera to a certain direction in case that the face is 
not recognized accurately, which gives advantage that the present 
invention may inteUigently photograph an accurate face without a user for 
the recognizing system moving to a designated position. 

The face photographing/ recognizing method and apparatus 
according to the present invention has been described in detail. However, " 
it should be understood that the detailed description and specific 
examples, while indicating preferred embodiments of the invention, are 
given by way of illustration only, since various changes and modifications 
within the spirit and scope of the invention will become apparent to those 
skilled in the art from this detailed description. 
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What is claimed is: 



1 . Method of photographing a face comprising: 

initial analyzing step for identifying p>osiuon of the face by sampling 
5 face data from image signals inputted from a camera installed in a 
predetermined position, and then selecting check area in order to track the 
identified face; 

pan /tilt or zoom control step for moving the camera- to right/ left, 
up/ down or forward /backward direction in order to track the identified 
10 face in the selected check area; 

determining step for determining whether the face is existing or 
abnormal by sampling and analyzing contour of the face identified in the 
check area; and 

storing and transmitting step for storing image of the determined 
15 face or transmitting the image to another recognizing system. 

2. Method of photographing a face as claimed in claim 1, further 
comprising the step of generating alarm about abnormal face after 
analyzing and determining the contour of the face sampled in the check 

20 area. 

3. Method of photographing a face as claimed in claim 2, wherein 
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the initial analyzing step comprises the steps of: 

detecting a moving portion by comparing a currently sampled image 
date with a previously stored image data; 

detecting a portion, in which proportions of color elements are in 
5 range of skin color, from the currently sampled image data; and 

selecting a portion where both skin color and motion are detected 
as a check area. 

4. Method for photographing a face as claimed in claim 3, 
10 wherein the initial analyzing step selects nearer image on the basis of a 

previously selected object for tracking as a new object for tracking in case 
that at least two faces are identiHed in the check area. 

5. Method for photographing a face as claimed in claim 4, 
15 wherein the initial analyzing step selects near image to center of 

photographing region as a new object for tracking in case that there is no 
previously selected object for tracking. 

6. Method for photographing a face as claimed in claim 2, 
20 wherein the determining step adjusts strength of the contour according to 

characteristics of person and/ or surroundings by sampling the contour 
of the face on the basis of average brightness of the image data in the 
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check area. 



7. Method for photographing a face as claimed in claim 6, 
wherein the determining step compares the contour of the image data 

5 sampled in the check area with a predetennined face template, and 
determines that the face exists in the check area when a contour 
approximates to the face template. 

8. Method for photographing a face as claimed in claim 7, 
10 wherein the determining step compares the predetermined face template 

with the contour in the check area by converting size of the predetermined 
face template to size of the check area. 

9. Method for photographing a face as claimed in, claim 7, 
15 wherein the determining step comprises the steps of: 

determining whether eyes are normal by partially retrieving the eyes 
after substituting the sampled face contour for a predetermined template 
around eyes: 

determining whether nose and mouth are normal by partially 
20 retrieving the nose and the mouth after substituting the sampled face 
contour for a predetermined template around nose and mouth, after 
completing the partial retrieving for the eyes; and 
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detennining whether the face is wholly normal by substituting 
numerical data of the sampled face contour-for predetermined numerical 
data of the eyes, the nose and the mouth, after completing the partial 
retrieving for the nose and the mouth. 

5 

10. Method for photographing a face as claimed in claim 2, 
wherein the face image storing and transmitting step retrieves and stores 
the face image among inputted data separately, 

10 11. Method for recognizing a face comprising: 

initial analyzing step for identifying position of the face by sampling 
face data from image signals inputted from a camera installed in a 
predetermined position, and then selecting check area in order to track the 
identified face; 

15 pan/ tilt or zoom control step for moving the camera to right/left, 

up/down or forward/backward direction in order to track the identified 
face in the selected check area; 

determining step for determining whether the face is existing or 
abnormal by sampling and analyzing contour of the face identified in the 
20 check area; and 

composing a personal mask by generating 3-dimensional 
information for a face image in the check area in case that the face is 
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existing, and then recognizing a specific person by comparing the personal 
mask with a previously stored personal mask. 

12. Method for recognizing a face as claimed in claim 11, further 
5 comprising the step of generating alarm about abnormal face after 

analyzing and determining the contour of the face sampled in the check 
area. 

13. Method for recognizing a face as claimed in claim 12, wherein 
10 the initial analyzing step comprises the steps of; 

' detecting a moving portion by comparing a currendy sampled image 
date with a previously stored image data; 

detecting a portion, in which proportions of color elements are in • 
range of skin color, from the currently sampled image data; and 
15 selecting a portion where both skin color and motion are detected 

as a check area. 

14. Method for recognizing a face as claimed in claim 13, wherein 
the initial analyzing step selects nearer image on the basis of a previously 

20 selected object for tracking as a new object for tracking in case that at 
least two faces are identified in the check area. 
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15. Method for recognizing a face as claimed in claim 14, wherein 
the initial analyzing step selects near image to center of photographing 
region as a new object for tracking in case that there is no previously 
selected object for tracking. 

16. Method for recognizing a face as claimed in claim 12, wherein 
the determining step adjusts strength of the contour according to 
characterisdcs of person and/or surroundings by sampling the contour 
of the face on the basis of average brightness of the image data in the 
check area. 

17. Method for recognizing a face as claimed in claim 16, wherein 
the determining step compares the contour of the image data sampled in 
the check area with a predetermined face template, and determines that 
the face exists in the check area when a contour approximates to the face 
template. 

18. Method for recognizing a face as claimed in claim 17, wherein 
the determining step compares the predetermined face template with the 
contour in the check area by converting size of the predetermined face 
template to size of the check area. 
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19. Method for recognizing a face as claimed in claim 17, wherein 
the determining step comprises the steps of: 

determining whether eyes are normal by partially retrieving the eyes 
after substituting the sampled face contour for a predetermined template 
5 around eyes: 

determining whether nose and mouth are normal by partially 
retrieving the nose and the mouth after substituting the sampled face 
contour for a predetermined template around nose and mouth, after 
completing the partial retrieving for the eyes; and 
10 determining whether the face is wholly normal by substituting 

numerical data of the sampled face contour for predetermined numerical 
data of the eyes, the nose and the mouth, after completing the partial 
retrieving for the nose and the mouth. 

15 20.. Method for recognizing a face comprising the steps of: 

selecting a plurality of check areas about image signals captured by 
a camera, selecting a specific check area by determining whether a face 
exists in the corresponding check area, and adjusting direction of the 
camera according to motion of an object in the selected check area; 

20 rearranging images by detecting characteristics about image signal 

where a face is existing in the check area after determining whether a face 
exists, generating 3-dimensional information according to displacement 
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between the rearranged images, and composing a face mask of the object 
with use of the 3-dimensional information; and - 

determining whether image for a specific person is registered or not 
by searching the composed face mask from personal registration masks 
in a database. 



2 1 . Method for recognizing a face comprising the steps of: 
selecting a plurality of check areas by respecdvely sampling skin 
color data and motion data from image signals inputted through a camera, 
selecdng a specific check area by retrieving a specific check area by 
inspecting face components about image signaJs in the corresponding 
check area, rearranging image by retrieving characteristics of the selected 
check area, and adjusting direction of the camera with determining 
whether a face exists or not with use of the rearranged image; 

rearranging images about image signals where a face is existing in 
the check area, generating 3-dimensionaI information according to 
displacement between the rearranged images, and composing a personal 
face mask by comparing the 3-dimensional information with a reference 
mask; and 

determining whether image for a specific person is registered or not 
by searching the composed face mask from personal registration masks 
in a database. 
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22. Apparatus for photographing a face comprising: 
image input means for capturing image of an object photographed 
by a camera installed in a predetermined position, and receiving serial 

5 image signals; 

pan/tilt or zoom driving means for moving the camera of the image 
input means to right/left, up/down or forward /backward direction; 

main control means for controlling the pan/tilt or zoom driving ■ • 
means such that the camera tracks motion of the object, selecting a check 
10 area by detecting skin color and motion of the object from the image 
signals inputted from the in.age input means, retrieving face portions from 
the image signals in the selected check area, and storing and displaying 

the face portions; 

storing or retrieving means for storing the image signals from the 
,5 main control means or retrieving face portion from the image signals; and 
display means for outputting the image signal corresponding to the 
face portion outputted from the main control means. 

23. Apparatus for photographing a face as claimed in claim 22. 
,0 further comprising alarm generating means for generating alarm according 
to results of face retrieval from the main control means. 
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24 Apparatus for photographing a face as claimed in claim 23, 
wherein the main control means comprises: - 

initial analyzing unit for selecting a check area by sampling portions 
which show skin color or motion from input images from the image input 
means; 

determining unit for sampling contour of the face from the check 
area, comparing the contour with a predetermined face template, 
determining whether the face is existing or abnormal according the 
comparison, and then selecting an object for tracking; 

object managing unit for determining whether the object for tracking 
selected by the determining unit is positioned in center of camera 
photographing region, and calculating and outputting a camera 
adjustment value of the camera for tracking motion of the corresponding 
object; 

pan/tilt or zoom control unit for generating pan/tilt or zoom control 
signal in order to move the image input meafis to a right/left, up/down or. 
forward/backward direction according to the camera adjustment value 
from the object managing unit; 

image storing and transmitting unit for storing face image captured 
by the image input means and transmitting the image to the display 
means; and 

face recognizing and retrieving unit for recognizing and retrieving the 
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stored face image. 

25. Apparatus for photographing a face as claimed in claim 24, 
wherein the initial analyzing iinit detects a moving portion by comparing 
a currently sampled image data with a previously stored image data, 
detects a portion, in which proportions of color elements are in range of 
skin color, and selects a portion where both skin color and motion are 
detected as a check area. 

26. Apparatus for photographing a face as claimed in claim 24, 
wherein the determining unit determines that a face exists in the check 
area when detecting a contour approximate to the face - template after 
comparing the detected contour with the previously stored face template. 

27. Apparatus for photographing a face as claimed in claim 26, 
wherein the determining unit compares the predetermined face template 
with the contour in the check area by converting size of the previously 
stored face template to the check area. 

28. Apparatus for photographing a face as claimed in claim 27, 
wherein, when selecting an object for tracking by comparing a currently 
sampled image data with the previously stored image data, in case that 
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two or more objects for tracking are detected, the determining unit selects 
an image most approximate to a previously selected object for tracking as 
a new object for tracking. - - 

5 29. Apparatus for photographing a face as claimed in claim 27, 

wherein, when selecting an object for tracking by comparing a currently 
sampled image data widi the previously stored image data, in case that 
two or more objects for tracking are detected and there is no previously 
selected object for tracking, the determining unit selects an image nearest 
10 to center of photographing region as a new object for tracking. 
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